Skip to content

Conversation

@Numpsy
Copy link
Contributor

@Numpsy Numpsy commented Jan 9, 2025

Description

As #2818, but updated on top of the latest changes and with a couple of extra commits to try to get more tests passing - see further comments for details

let scenarioPath = resolvePath scenario ""
// dotnet tool install --version 5.19.0-alpha.local.1 fake-cli --add-source /e/Projects/FAKE/release/dotnetcore/

// Work around https://github.com/dotnet/sdk/issues/40655 by specifying the tool manifest explicitly
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of tests in previous runs seemed to be failing with dotnet/sdk#40655, whereby installing the dotnet tool in a subdirectory that contains a tools config file will ignore that file and use the one in the repo root, but then trying to use that tool will use the local config file and then fail.
That issue is marked as resolved, but it still seemed to be occuring in the SDK version used by the tests

|> Async.RunSynchronously
|> List.ofSeq
|> List.find (fun product -> product.ProductVersion.Equals("6.0"))
|> List.find (fun product -> product.ProductVersion.Equals("8.0"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the logic for this (or cachedDotnetSdkReleases.json) should be when updating .NET versions.
The current cachedDotnetSdkReleases.json only contains the .NET 6 releases and that seemed to cause test failures when built as .NET 8 and looking for the v8 SDK versions.
Changing this to 8.0 fixed that, but that might then not be right if fake-cli is multi-targetted at .NET 6 and .NET 8 as the .NET 6 version might still need the list of v6 releases?

@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025

Test Results

   12 files  ±0     12 suites  ±0   32m 54s ⏱️ +18s
  451 tests ±0    450 ✅ ±0  1 💤 ±0  0 ❌ ±0 
1 287 runs  ±0  1 284 ✅ ±0  3 💤 ±0  0 ❌ ±0 

Results for commit ae027fd. ± Comparison against base commit 74e1e6f.

This pull request removes 3 and adds 3 tests. Note that renamed tests count towards both.
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 6.0.300 SDK version assemblies and resolve runtime version from cached file]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 6.0.300 SDK version assemblies]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 6.0.301 SDK version assemblies]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.400 SDK version assemblies and resolve runtime version from cached file]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.400 SDK version assemblies]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.401 SDK version assemblies]

♻️ This comment has been updated with latest results.

@Numpsy
Copy link
Contributor Author

Numpsy commented Jan 9, 2025

The remaining failures are in the TemplateIntegrationTests, with errors like

25h[11:48:10 ERR] Fake.DotNet.Cli.IntegrationTests.TemplateTests.can install and run the template.fails to build a target that doesn't exist failed in 00:00:09.6230000. 
Expected the message 'Target Nonexistent is not defined' but got: No exact match of product releases 8.0.401 found.
No product release found for 8.0.401. Maybe a pre-release? Returning all the versions.
No exact match of product releases 8.0.404 found.
No product release found for 8.0.404. Maybe a pre-release? Returning all the versions.
There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: '[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.36\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.35\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.33\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.32\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.31\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.30\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.29\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.28\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.27\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.26\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.25\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.24\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.23\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.22\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.21\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.20\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.19\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.18\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.16\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.15\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.14\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.13\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.12\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.11\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.10\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.9\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.8\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.7\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.6\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.4\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.3\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.2\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.1\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-rc.2.21480.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-rc.1.21451.13\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.7.21377.19\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.6.21352.12\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.5.21301.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.4.21253.7\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.3.21201.4\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.2.21154.6\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.1.21102.12\ref\net6.0]', please check installed SDK and runtime versions

and I'm not sure if this is really a problem with the new stuff, or if it's because it's trying to use the 6.1.3 release to test the templates, and failing when running as .NET 8? (I suppose it shou;dn't be looking at the 6.0 assemblies when running as 8.0)

@Thorium
Copy link
Member

Thorium commented Jul 17, 2025

I expect that there is a chicken-and-egg problem, and the integration tests are actually restoring the .NET6 fake to try compiling .NET8, instead of referring to the just built .NET 8 version.

I tried this branch locally (Numpsy/new8-build-bump-roll-tests-2) on my machine, and my observations are here:

src\app\fake-cli\bin\Release\ has now net6.0 and net8.0 paths. Can a dotnet tool even have multiple target frameworks? Or should we just accept that FAKE 6.1.3 is working well for .NET6 projects and from now on everything else is .NET 8 only?

I run on my Windows machine dotnet build, and then from the actions:
dotnet tool install fake-cli --add-source "./src/app/fake-cli/bin/Debug" --add-source "./src/app/fake-cli/bin/Release" --version 1.0.0-1

It turned out that this didn't work properly. So I removed the Clean-task and called fake manually:

C:\git\FAKE>src\app\fake-cli\bin\Debug\net8.0\fake-cli -v build -e FAKE_DETAILED_ERRORS=true

And behold, all the integration tests went through, after painfully long half-an-hour later:

image

Where it did finally fail is:

dotnet.exe  deb --runtime linux-x64 --framework net8.0 --configuration Release --output "C:\git\FAKE\release\dotnetcore" --no-restore`

error NETSDK1045: The current .NET SDK does not support targeting .NET 8.0.  Either target .NET 6.0 or lower, or use a version of the .NET SDK that supports .NET 8.0. [C:\git\FAKE\src\app\fake-cl

So basically, dotnet deb tool has to be updated from 0.1.220 to version 0.1.232

"version": "0.1.220",

...and that's it, I think we could consider merging this and releasing a beta or new major version.

@xperiandri
Copy link
Collaborator

Can a dotnet tool even have multiple target frameworks?

@Thorium yes it can.
But there are some challenges with that.
You can see my changes in this PR to test that the older framework works fsprojects/FSharpLint#748

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

well for .NET6 projects and from now on everything else is .NET 8 only?

Whether there's any benefit to supporting .NET 6 in fake-cli any more is another question, and given the amount of pain in testing it all, just doing 8 seems simpler for a new major release?

.NET 10 is on the horizon, so making sure it works for that going forward seems more important than keeping 6 working in new versions. (My personal opinion anyway).

I'll see about rebasing it and fixing the conflicts anyway

@Thorium
Copy link
Member

Thorium commented Jul 17, 2025

Many projects use a separate non-FAKE fsproj as a build-project and reference the FAKE libraries, and there I see value in having .NET6 support kept.

But the fake-cli tool, I think, could target only the NET8 (or even 10 later).

@xperiandri
Copy link
Collaborator

xperiandri commented Jul 17, 2025

Then it will be impossible to run it unless you have the latest SDK installed

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

Then it will be impossible to run it unless you have the latest SDK installed

The 'DNX' stuff that MS are talking about with .NET 10 (https://github.com/dotnet/core/blob/main/release-notes/10.0/preview/preview6/sdk.md#platform-specific-net-tools) could maybe help if the tool could be self contained, but that's a work in progress so it remains to be seen

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

On the subject of targetting multiple .NET versions in fake-cli, re-reading the old comments reminded me of the situation with cachedDotnetSdkReleases.json - i.e., if the tool i smulti-targetted, you might need multiple versions of that file for different .NET versions?

@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 0364d81 to 233081b Compare July 17, 2025 17:31
@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

@Thorium I didn't see your other PR until after i'd rebased the changes to fix the merge conflicts, which caused conflicts in that, so I've cherry picked your dotnet-deb update straight into this PR

@Numpsy Numpsy marked this pull request as ready for review July 17, 2025 19:07
@Thorium
Copy link
Member

Thorium commented Jul 18, 2025

I think this file https://github.com/fsprojects/FAKE/blob/master/src/template/fake-template/Content/.config/dotnet-tools.json
says "use the nuget version of FAKE in the integration test templates", which causes any major changes to FAKE (like .NET update) to break the integration tests, because the NuGet version doesn't have these changes yet.

But I'm not sure if there is any workaround possible.

@Thorium
Copy link
Member

Thorium commented Jul 18, 2025

One more thing: Because the system uses FSharp.Compiler.Service v43.8, the fsdocs-tool has to be upgraded from 16 to 20, to make GenerateDocs task pass.

Edit: Updated my PR, but you can also just do dotnet tool update fsdocs-tool instead.

After this, my local build does pass:
image

@xperiandri
Copy link
Collaborator

@arialdomartini

@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 62084b1 to 7b0784d Compare October 21, 2025 10:09
@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 7b0784d to 41a04b5 Compare October 25, 2025 21:42
@Numpsy
Copy link
Contributor Author

Numpsy commented Oct 26, 2025

General question about all the FAKE modules - should they all be both .NET 6 and .NET 8, or just NET 8? (or maybe only 6 if they don't have any dependencies where it matters?)

@Numpsy
Copy link
Contributor Author

Numpsy commented Oct 26, 2025

Otherwise the basic question is whether this should be turned into 8.0.0 alpha 1 and then the templates and tests fixed on top of that, or if it's worth trying something more minimal with just a .NET 8 / v8 build of fake-cli and then doing the modules afterwards.

@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 41a04b5 to ba8e1ed Compare October 26, 2025 13:47
@Numpsy
Copy link
Contributor Author

Numpsy commented Oct 26, 2025

I expect that there is a chicken-and-egg problem, and the integration tests are actually restoring the .NET6 fake to try compiling .NET8, instead of referring to the just built .NET 8 version.

I tried this branch locally (Numpsy/new8-build-bump-roll-tests-2) on my machine, and my observations are here:

src\app\fake-cli\bin\Release\ has now net6.0 and net8.0 paths. Can a dotnet tool even have multiple target frameworks? Or should we just accept that FAKE 6.1.3 is working well for .NET6 projects and from now on everything else is .NET 8 only?

I run on my Windows machine dotnet build, and then from the actions: dotnet tool install fake-cli --add-source "./src/app/fake-cli/bin/Debug" --add-source "./src/app/fake-cli/bin/Release" --version 1.0.0-1

It turned out that this didn't work properly. So I removed the Clean-task and called fake manually:

C:\git\FAKE>src\app\fake-cli\bin\Debug\net8.0\fake-cli -v build -e FAKE_DETAILED_ERRORS=true

And behold, all the integration tests went through, after painfully long half-an-hour later:
image

Where it did finally fail is:

dotnet.exe  deb --runtime linux-x64 --framework net8.0 --configuration Release --output "C:\git\FAKE\release\dotnetcore" --no-restore`

error NETSDK1045: The current .NET SDK does not support targeting .NET 8.0.  Either target .NET 6.0 or lower, or use a version of the .NET SDK that supports .NET 8.0. [C:\git\FAKE\src\app\fake-cl

So basically, dotnet deb tool has to be updated from 0.1.220 to version 0.1.232

"version": "0.1.220",

...and that's it, I think we could consider merging this and releasing a beta or new major version.

It appears to work (all integration tests pass) if the .NET 6 runtime files are present in the local .NET install created by the tests, but not otherwise.

So

If i run the tests from this branch on a clean setup, if fails as the CI does.

If I run the build from the master branch, it installs the .NET 6 libs into %AppData%\Local\Microsoft\dotnet to run the .NET 6 tests, and leaves the files there.

If I then run the tests from this branch again, it installs the .NET 8 libs into %AppData%\Local\Microsoft\dotnet alongside the .NET 6 libs rather than replacing them, and the template integration tests find those and the tests pass.

Having .NET 6 installed in the default system location doesn't effect this as the tests are looking in the custom install location.

@SteveGilham
Copy link
Contributor

As net6.0 is past end of support, as is net7.0, after any necessary bridging release to get to net8.0, it would make sense to drop all mention of net6.0 - and work to prepare for net10.0 so we don't have an end of support crisis this time next year as both net8.0 and net9.0 are superseded on 10-Nov-26.

Let us hope that that process that time can be made as easy as "for a bridging release add mention of net10.0 at these points in the build system, the project targets and the code, then enable this bridging flag; for real net10.0 release, remove net8.0 and clear the flag. Lather, rinse repeat for net12.0, net14.0 ..."

@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch 2 times, most recently from fe22252 to 21017b8 Compare October 27, 2025 20:37
@Numpsy
Copy link
Contributor Author

Numpsy commented Oct 27, 2025

So, I tried changing the template tests to create a test local nuget.config file which points at the locally built nuget packages, such that the tests use the local v8 alpha build instead of using the 6.1.4 builds from nuget.org, and that has made the tests pass.

"args": {
"executable": "dotnet",
"args": "tool update fake-cli"
"args": "tool update fake-cli --prerelease"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the tests needs the --prerelease here otherwise it tries to replace the v8 alpha build with the released 6.1.4 build and fails.
I think it's fine for the v8 alpha template to have prerelrease enabled, as long as it's changed before the final release

<packageSources>
<clear />
<add key="api.nuget.org" value="https://api.nuget.org/v3/index.json" />
<add key="localBuild" value="{nugetPackageFolder}" />
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows the tests to find the locally built v8-alpha fake-cli package instead of always using the 6.1.4 version off nuget.org, which lets the test pass by using the .NET 8 version instead of the .NET 6 version

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although RuntimeAssemblyVersions now defaults to a 2-element list, this is actually pointless as everything derived from the value starts by goes through |> Seq.head - lines 65-68 of this file. Retaining the sequence nature may help with the net10.0 (and future) update.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 283, 295 still refer to a method net60releases; this could do with renaming when it refers to net8.0 SDKs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it should be both 6/8 now, or just 8 (will something running on .NET 8 ever want the .NET 6 libs?)
@Thorium ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe line 51 should be ReleaseVersion "8.0.0" as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 283, 295 still refer to a method net60releases; this could do with renaming when it refers to net8.0 SDKs.

renamed to netReleases

Thorium and others added 14 commits October 29, 2025 09:38
Change global.json .NET 6 to 8
Search and replace targetframeworks from fsproj files to add net8.0
Add net8.0 to paket.dependencies
dotnet paket install to find .NET8 compatible dependencies
Expecto had to be hardcoded for now, because some tests are running on netstandard2.0 library (hopefully we can update this separately later)
MSBuild.StructuredLogger problem: DisableInternalBinLog = true had to be added to build.fsx NuGet commands (hopefully we can update this separately later)
A few places of code had new overrides so had to explicitly type to strings
SdkAssemblyResolver to default .NET 8 as well
Readme update
GitHub pipeline configs: Add .NET 8 install.
…l has to be upgraded from 16 to 20, to make GenerateDocs task pass.
…the 8.0.0 alpha build

Instead of pulling the 6.1.X build off NuGet.org.
This might bootstrap the tests enough to get them all passing
@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 21017b8 to 439ba10 Compare October 29, 2025 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants